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Abstract 

Assume that a stochastic process can be approximated, when some scale parameter 
gets large, by a fluid limit (also called "mean field limit", or "hydrodynamic limit"). 
A common practice, often called the "fixed point approximation" consists in approx- 
imating the stationary behaviour of the stochastic process by the stationary points 
of the fluid limit. It is known that this may be incorrect in general, as the stationary 
behaviour of the fluid limit may not be described by its stationary points. We show 
however that, if the stochastic process is reversible, the fixed point approximation 
is indeed valid. More precisely, we assume that the stochastic process converges to 
the fluid limit in distribution (hence in probability) at every fixed point in time. 
This assumption is very weak and holds for a large family of processes, among which 
many mean field and other interaction models. We show that the reversibility of the 
stochastic process implies that any limit point of its stationary distribution is concen- 
trated on stationary points of the fluid limit. If the fluid limit has a unique stationary 
point, it is an approximation of the stationary distribution of the stochastic process. 
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1. Introduction 

This paper is motivated by the use of fluid limits in models of interacting objects 
or particles, in contexts such as communication and computer system modelling [B], 
biology [7J or game theory [3]. Typically, one has a stochastic process Y N , indexed by 
a size parameter N; under fairly general assumptions, one can show that the stochas- 
tic process Y N converges to a deterministic fluid limit (p [17]. We are interested in the 
stationary distribution of Y N , assumed to exist and be unique, but which may be too 
complicated to be computed explicitly. The "fixed point assumption" is then some- 
times invoked [T5l [221 EH] : it consists in approximating the stationary distribution 
of Y N by a stationary point of the deterministic fluid limit ip. In the frequent case 
where the fluid limit <p is described by an Ordinary Differential Equation (ODE), say 
of the form y = F(y), the stationary points are obtained by solving F(y) = 0. If Y N 
is an empirical measure, convergence to a deterministic limit implies propagation of 
chaos, i.e. the states of different objects are asymptotically independent, and the dis- 
tribution of any particular object at any time is obtained from the fluid limit. Under 
the fixed point assumption, the stationary distribution of one object is approximated 
by a stationary point of the fluid limit. 

A critique of the fixed point approximation method is formulated in [2J, which 
observes that one may only say, in general, that the stationary distribution of Y N 
converges to a stationary distribution of the fluid limit. For a deterministic fluid 
limit, a stationary distribution is supported by the Birkhoff center of the fluid limit, 
which may be larger than the set of stationary points. An example is given in [2J 
where the fluid limit has a unique stationary point, but the stationary distribution 
of Y N does not converge to the Dirac mass at this stationary point; in contrast, it 
converges to a distribution supported by a limit cycle of the ODE. If the fluid limit 
has a unique limit point, say y*, to which all trajectories converge, then this unique 
limit point is also the unique stationary point and the stationary distribution of Y N 
does converge to the Dirac mass at y* (i.e. the fixed point approximation is then 
valid). However, as illustrated in [2J, this assumption may be difficult to verify, as it 
often does not hold, and when it does, it may be difficult to establish. For example, 
in [9] it is shown that the fixed point assumption does not hold for some parameter 
settings of a wireless system analyzed in [5], due to limit cycles in the fluid limit. 

In this paper we show that there is a class of systems for which such complications 
may not arise, namely the class of reversible stochastic processes. Reversibility is 
classically defined as a property of time reversibility in stationary regime [13]. There 
is a large class of processes that are known to be reversible, for example product- 
form queuing networks with reversible routing, or stochastic processes in [TJ], which 
describes the occupancy of inter-city telecommunication links; in Section [5] we give 
an example motivated by crowd dynamics. In such cases, we show that the fluid limit 
must have stationary points, and any limit point of the stationary distribution of Y N 
must be supported by the set of stationary points. Thus, for reversible processes that 
have a fluid limit, the fixed point approximation is justified. 
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2. Assumptions and Notation 

2.1. A Collection of Reversible Random Processes 

Let E be a Polish space and let dbea measure that metrizes E. Let V(E) be the 
set of probability measures on E, endowed with the topology of weak convergence. 
Let Cb(E) be the set of bounded continuous functions from E to R, and similarly 
Cb(E x E) is the set of bounded continuous functions from E x E to R. 

We are given a collection of probability spaces (Q N , F N ,F N ) indexed by N = 
1,2,3, ... and for every N we have a process Y N defined on (Q N , F N ,F N ). Time is 
continuous. Let De[0, oo) be the set of cadlag functions [0, oo) — > E\ Y N is then a 
stochastic process with sample paths in De[0, oo). 



We denote by Y (t) the random value of Y N at time t > 0. Let E N C E be the 
support of 1^(0), so that ¥ N (Y N (0) e E N ) = 1. 



We assume that, for every N, the process Y is Feller, in the sense that for every 
t > and h G C b (E), E N [h(Y N (t))\ Y N (0) = y ] is a continuous function of y G E. 
Examples of such processes are continuous time Markov chains as in [TB], or linear 
interpolations of discrete time Markov chains as in j3], or the projections of a Markov 
process as in [12]. 



Definition 1. A probability H N G V(E) is invariant for Y N if U N (E N ) = 1 and for 
every h G Cb(E) and every t > 0: 



We are interested in reversible processes, i.e. processes that keep the same sta- 
tionary law under time reversal. A weak form of such a property is defined as follows 

Definition 2. Assume II is a probability on E such that U N (E N ) = 1, for some N. 
We say that Y N is reversible under II if for every time t > and any h £ Cb(E x E): 



E N [h {y, Y N (t)) | Y N {0) = y] U N (dy) — E N [h (Y N (t), y) \ Y N (0) = y] Il N (dy) 



Note that, necessarily, IT is an invariant probability for Y . If Y is an ergodic 
Markov process with enumerable state space, then Definition [2] coincides with the 
classical definition of reversibility by Kelly in [13]. Similarly, if Y N is a projection of 
a reversible Markov process X N , as in [TO] , then Y N is reversible under the projection 
of the stationary probability of X N ; note that in such a case, Y N is not Markov. 
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2.2. A Limiting, Continuous Semi-Flow 

Further, let <p be a deterministic process, i.e. a measurable mapping 



<p : [0, oo) x E 

t,yo 



E 

*-> <Pt{yo) 



We assume that ip t is a semi-flow, i.e. 

i- ^o(y) = y, 

2. <p s +t = V^s ° V 9 * f° r all s > and t > 0, 
and we say that 90 is "space continuous" if for every £ > 0, <pt(y) is continuous in y. 

Definition 3. We say that y G E is a stationary point of if ^? t (t/) = 2/ for all £ > 

In cases where E is a subset of M. d for some integer d, the semi-flow may be an 
autonomous ODE, of the form y = F(y); here the stationary points are the solutions 
of F(y) = 0. 

Definition 4. We say that the semi-flow tp is reversible under the probability n e 
V(E) if for every time t > and any h £ Cb(E x E): 



As we show in the next section, reversible semi-flows must concentrate on stationary 
points. 

2.3. Convergence Hypothesis 

We assume that, for every fixed t the processes Y N converge in distribution to 
some space continuous deterministic process as iV — > 00 for every collection of 
converging initial conditions. More precisely: 

Hypothesis 1. For every yo in E, every sequence (jJq )jv=i,2,... such thaty§ G E and 
limTv^oo ?/q = y , and every t > 0, the conditional law ofY N (t) given 1^(0) = y^ 
converges in distribution to the Dirac mass at tp t (yo). That is 



for all h G Ct,(E) and any fixed t > 0. In the above, ip is a space continuous semi-flow. 




(1) 



lim E N [h(Y N (t))\Y N (0) = jff] = h o ^ t {y ) 



(2) 
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Hypothesis [T] is commonly true in the context of fluid or mean field limits. The 
stronger convergence results results in [T8J [HI [2U E| imply that Hypothesis [1] is 
satisfied ; we give a detailed example illustrating this in Section [5j Similarly, [2] gives 
very general conditions (called HI to H5) that ensure convergence of a stochastic 
process to its mean field limit; under these conditions, Hypothesis [1] is automatically 
satisfied (the deterministic process tp is then an ODE). Note that the results in these 
references are stronger than what we require in Hypothesis [TJ for example in [16] 
there is almost sure, uniform convergence for all t G [0, T], for any T > 0; in [12] the 
convergence is on the set of trajectories. 

Under Hypothesis [[J ip is called the hydrodynamic limit or simply fluid limit of 

3. Reversible Semi-Flows Concentrate on Stationary Points 

Theorem 1. Let p be a space continuous semi- flow, reversible under U. Let S be the 
set of stationary points of p. Then U is concentrated on S, i.e. H(S) = 1. 

Proof. 

Step 1. Denote with S c the complement of the set of stationary points. Take 
some fixed but arbitrary y G S c . By definition of S, there exists some r > such 
that 

<^2rM ^ Vo (3) 

Define p T (y ) = y x , pAyi) = 2/2, so that y 2 ^ y . 

For y G E and e > we denote with B(y, e) the open ball = {x G E, d(x, y) < e}. 
Let e = d(yo, y 2 ) > and let B 2 = B(y 2 , e/2). Since the semi-flow is space continuous, 
there is some a>i > such that B\ = B(i)\,a\) and <p T (Bi) C B 2 . Also let B[ = 
B(yi,ai/2). By the same argument, there exists some ao > such that ao < e/2, 
B = B(y , ao) and p T (B ) C B[. We have thus: 

Pr(B ) C B' l dB l 
Pr{B{) C B 2 
B n B 2 = 

Let ^ be some continuous function [0, +oo) — > [0,1] such that £(w) = 1 whenever 
< u < 1/2 and £(w) = whenever u > 1 (for example take a linear interpolation). 
Now take 

h{y ^^d^\/d^\ 
\ a J \ ai J 

so that h G Cb(E x E) and 

h(y, z) — whenever y ^ B oi z ^ B\ 

h(y, z) — 1 whenever rf(y , y) < a /2 and z G -Bj 
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It follows that h(ip T (z), z) = for every z G E and 



/ h(y, ip T (y))U(dy) > U(B(y , a /2)) (5) 

J E 

Apply Definition HI it comes II (-B(j/o, ctio/2)) = 0; thus, for any non stationary point 
yo there is some a > such that 

n (^(j/o, a)) = (6) 

Step 2. The space is polish thus also separable, i.e. has a dense enumerable set, 
say Q. 

For every y <E S c let a be as in Eq.(j6]) and pick some q(y) G Q and n(y) G N s.t. 
d(j/, q(y)) < 4) < «. Thus y G fl(g(j/), ^) and n (5(g(j/), ^)) = 0. 
Let F = U y es c (<?(?/)' n (v))- F C Q x N thus F is enumerable and 



sc - u 

Z' \s- TP V / 



(g,n)eF 

Thus 



(g,n)GF 



(7) 



□ 



Note that it follows that a semi-flow that does not have any stationary point 
cannot be reversible under any probability. 



4. Stationary Behaviour of Fluid Limits of Reversible Processes 

Theorem 2. Assume for every N the process Y N is reversible under some probability 
U N . Assume the convergence Hypothesis U\ holds and that IT G V(E) is a limit point 
(for weak convergence) of the sequence LT . Then the fluid limit is reversible under IT. 
In particular, it follows from Theorem[l\that U is concentrated on the set of stationary 
points S of the fluid limit <p. 



Proof. All we need to show is that II verifies Definition HI Let Nk be a subsequence 
such that lim/c-j.oo IT^ = II in the weak topology on V(E). By Skorohod's representa- 
tion theorem for Polish spaces [Til Thm 1.8], there exists a common probability space 
(Q, J 7 , P) on which some random variables X k for k G N and X are defined such that 

law of X k = U Nk 
law of X = II 
X k -»XP-a.s. 
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Fix some t > and h £ C&(i? x £"), and define, for £ N and y £ E 

a k (y) ^ E(/i(y,Y"*(f))|Y"*(0)=i/) 
= E(h(Y N *(t),y)\Y N >(0)=y) 

Since is reversible under IT^: 

/ a k (y)U Nk (dy) = [ b k (y)U Nk (dy) (8) 

J E J E 

Hypothesis [1] implies that lim^oo a k (x k ) = h(x,(p t (x)) for every sequence x k such 
that x k £ E Nk and Hm.k^ f00 x k = x £ E. Now X fe £ E Nk P— almost surely, since the 
law of X k is H Nk and Y Nk is reversible under IT^ . Further, X k — > X P— almost 
surely; thus 

lim a k (X k ) = h(X,tp t (X)) P - almost surely (9) 

k— ¥oo 

Now a k (X k ) < \\h\\ and, thus, by dominated convergence: 

lim E (a k (X k )) = E (h(X, ipt(X))) (10) 

and similarly for b k . Thus 

/ h(y,tp t (y))IL(dy)= [ h(<p t (y) , y)U(dy) (11) 

J E J E 

□ 

In particular, if the semi-flow has a unique stationary point, we have: 

Corollary 1. Assume the processes Y N are reversible under some probabilities Tl N . 
Assume Hypothesis U\ holds and: 

1. the sequence (JI n )n=i,2,... is tight; 

2. the semi-flow tp has a unique stationary point y* . 

It follows that the sequence U N converges weakly to the Dirac mass at y* . 

We leave the proof of the corollary to the reader (it follows in a classical way from 
compactness arguments; the tightness condition implies that the set (11^, N = 1,2, 3... } 
is relatively compact in V{E)). 

Recall that tightness means that for every e > there is some compact set K C E 
such that U N (K) > 1 — e for all N. If E is compact then (H n )n=i,2,... is necessarily 
tight, therefore condition 1 in the corollary is automatically satisfied. For mean field 
limits where E is the simplex in finite dimension, the corollary says that, if the pre- 
limit process is reversible, then the existence of a unique stationary point implies that 
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the Dirac mass at this stationary point is the limit of the stationary probability of 
the pre-limit process. 

Compare Corollary [T] to known results for the non reversible case pQ: there we 
need that the fluid limit tp has a unique limit point to which all trajectories converge. 
In contrast, here, we need a much weaker assumption, namely, the existence and 
uniqueness of a stationary point. It is possible for a semi-flow to have a unique 
stationary point, without this stationary point being a limit of all trajectories (for 
example because it is unstable, or because there are stable limit cycles as in [2]). In 
the reversible case, we do not need to show stability of the unique stationary point 

y*- 

5. Example: Crowd Dynamics 

In this section we give an example to illustrate the application of Theorem [2] - 
a detailed study of this example beyond the application of Theorem [2] is outside the 
scope of this paper. We consider the crowd dynamics model of [23]. The model 
captures the emergence of crowds in a city. A city is modelled as a fully connected 
bidirectional graph with I vertices, every vertex representing a square, where bars are 
located. There is a fixed total population N. People spend some time in a square and 
once in a while decide to leave a square and move to some other square. The original 
model is in discrete time, and at every time slot, the probability that a tagged person 
present in square i leaves square i is assume to be equal to (1 — c)^*® -1 where iVj(i) 
is the population of square % at time t. In this equation, c is the chat probability, 
and this model thus assumes that a person leaves a square when it has no one to 
chat with. The model also assumes that departure events are independent. When a 
person leaves a square i, it move to some other square j according to Markov routing, 
with probability Qij given by 

Q -' - w, (12) 

where d(i) is the degree of node i, i.e. the person picks a neighboring square j 
uniformly at random among all neighboring squares. 

In [23], the authors study by simulation the emergence of concentration in one 
square. They also show that for regular graphs (i.e. when all vertices have same 
degree) there is a critical value c* such that for c > c* concentrations occur, whereas 
for c < c* the stationary distribution of people is uniform. The analysis is based on 
the study of stationary points for the empirical distribution. Note that, as mentioned 
in the introduction, the analysis with stationary points may, in general, miss the 
main part of the stationary distribution, and it is quite possible that the stationary 
distribution is not concentrated on stationary points (for example if there is a limit 
cycle [2]). A fluid flow approximation is proposed in [21], and similar results are 
found. 

To understand whether the stationary point analysis is justified, we study the 
large N asymptotics for an appropriately rescaled version. To avoid unnecessary 



S 



complications, we replace the original model of [23], which is in discrete time, by its 
continuous time counterpart. The probability that a tagged person present in square 
i leaves square % in a time slot is replaced by the rate of service given by 

fjL N (Ni) = (1 - cf*- 1 (13) 

i.e. the probability that a tagged person present in square % leaves the square in the 
next dt seconds is fi N {N^dt + o(dt). 

The corresponding continuous process X N (t) = (Ni(t), Ni(t)) is a Markov 
process on an enumerable state space. More precisely, it is a queuing network of 
infinite server stations, with state dependent service rate and with Markov routing. 
It follows from classical results on quasi-reversibility that it has product-form (see 
for example [191 Chapter 8]), i.e. it is ergodic (since the graph of squares is fully 
connected and the population is finite) and its stationary probability is given, for 
every (ni, ni) G M J such that n\ + ... + nj = N by 

P N (n l ,...,n I ) = rf ft f (14) 

i=i 

In this formula, rj N is a normalizing constant, ff{n) = f % Ilm=i Z 1 "^)) an d the 
vector 9 = (9\, Oj) is the stationary distribution of the Markov routing matrix Q, 
i.e. a normalized solution of the equation 9Q = 6. Note that it follows from Eq.( )12j) 
that 

B t = -4^- (15) 

To apply Theorem [2l we need to show that X N is reversible. A product-form 
queuing network is, in general, not reversible. However, it is so if the Markov routing 
chain is reversible [20], which is the case here. 



Theorem 3. For every N , the process X N is reversible. 

Proof. Take 9 given by Eq. (fT5l) . Then 9iQij = 9jQj t i for any pair thus the 

Markov chain with transition matrix Q given by Eq. (jl2p is reversible. By [2U], it 

follows that the product-form queuing network X N is reversible. □ 

In [8], it is suggested to scale the chat probability as 

(16) 

in order to account for the fact that, for large populations, meetings tend to be 
limited by space or size of the friend's group. We use this scaling law and consider 
the re-scaled process Y N of occupancy measures, i.e. given by 

( N M N i(t)\ i vN , 



y "w = lir--ir = n x (t) (17) 
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Obviously, for every iV the process Y N is Markov and is reversible. Further, it 
converges to an ODE, as we see next. 

To establish the convergence of Y , we compute its drift 

g Um e ( + *)- „| y w = 

vyy dt^o (it v 1 

defined for every possible value y of Y N (t). When the occupancy measure is y, 
there are iVj = iVy» people in square i, and the rate of departure from square % is 
Nifj,(Ni) = Nyi/j,(Nyi)] the delta to the occupancy measure due to one person moving 
from square % to square j is ~ e ^ 6j , where is the row vector with a 1 in position i 
and elsewhere. Therefore 

V N (y) = J^Ny^NyJQ^^pi- 
Taking into account Eq. (lToT) . it comes 

V N (y) = (l " ^Y^ 1 Qv + e i) (19) 

y 

Let A/ denote the simplex, i.e. 

Aj = jy G M 7 , j/j > for all i and E jfc = 1 

When N — > oo, V JV (?/) converges for every y G A/ to 

V(2/) = E J/i e ~ W ^ (-^ + e i) ( 2 °) 

y 

This suggests that the fluid limit of y , if it exists, would be the deterministic process 
y(t), with sample paths in M. d , obtained as solution of the ODE % = V(y). As we 



show next, this is indeed the case and follows from "Kurtz's theorem" [24] Theorem 
9.2.1]. Before that, we rewrite the ODE more explicitly as 

dyi 



dt 



E •'//'' *" :( h-> (21) 



Theorem 4. Assume that the initial condition y$ of the process Y N is deterministic 
and converges to some y G Aj. Let tp be the semi-flow defined by the ODE l[21\) . i.e. 
ft(yo) ^ the solution of the ODE l[21\) with initial condition y(0) = y (this solution 
exists and is unique by the Cauchy Lipschitz theorem). Then for each T > and 
e > 0: 



lim P sup r v (t) - ip t (y )\\ > e = (22) 

JV-KX) \o<t<T 



(the notation \\\\ stands for any norm on M. 1 ). 
It follows that Hypothesis 1 is verified. 
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Proof. We apply Theorem 9.2.1 in [23]. We need to find a sequence of numbers 
5yv — > such that the following three conditions hold: 



lim sup \\V N (y) - V(y)\\ = 
sup sup A (y) < oo 
hm sup A? N (y) = 



(23) 
(24) 
(25) 



In the above, A^ is the set of feasible states of Y N , i.e. the set of y G Aj such that 
Ny is integer, A (y) is the expected norm of jump per time unit, and Af (y) is the 
absolute expected norm of jump per time unit due to jumps travelling further than 
S N . 

We now show Eq. (l23j) . First consider the case y G such that yi > (thus we 
have 1/N <yi< 1). We apply the inequality \e~ x — e~ x [ < \x — x'\, valid for x > 
and x' > 0, to x = — (Nyi — 1) log (l — , x' — sy, and obtain: 



1 ~n) 



S \Nyi-l 



< 



(N Vi - 1) log ( 1 - - 



The right handside is convex in yi thus its maximum for yi G [1/N, 1] is obtained at 
one end of the interval. Thus 



1 



S \Nyi-l 

NJ 



— e 



-syi 



/ \ dcf 

< aN{s) = max 



s 

iV' 



(N - 1) loj 



N 



Second, multiply by yi and note that yi < 1, it follows that, whenever y G A^ and 
Vi > 0: 

s 



Vi i 



< ajv(s 



and this is also obviously true if = 0. It follows that 

sup \\V N (y)-V(y)\\<a N (s)y2\\-e i + e J \\Q 



from where Eq. (l23p follows since limTv-i-oo a 7v( s ) = 0. 

We now show Eq. fl24p . We take the sup norm on M. d so that ||— 
i 7^ j; thus 



1 for 



A N (y) 



dcf 



E4 1 



'■./ 



S \Nyt-l 

n) 

-V 

N) 
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|| &i ~\~ &j[ 



I.J 



which shows Eg. (1241) . 

We now show Eq. (!25|) . We take 5n = j?- The jumps of Y N are of the form 



and thus || — jf^-\\ < <5yv- Thus 



which trivially shows Eg. ( 125]) . By Theorem 9.2.1 in [21], this establishes Eq.(l2~2l). 

It follows (as a much weaker convergence) that for any fixed T, Y (T) converges 
in probability to the deterministic yr(z/o)- Thus there is also convergence in law, 
since convergence in probability to a deterministic variable implies convergence in 
distribution, i.e. Eq.(j2]) in Hypothesis 1 is verified. It remains to see that ip t is space 
continuous: this follows from the fact that the right-handside of the ODE is Lipschitz 
continuous and from the Cauchy Lipschitz theorem. □ 

It follows from Theorem |2]that any limit point of the stationary probability Eq.( ll4"|) 
is concentrated on the stationary points of the ODE (liZTj) . This justifies a posteriori 
the method in [23], which looked only at stationary points. 

For the case of a regular graph (this is the case studied analytically in [23]). 
the stationary points can be obtained explicitly (Theorem 6.1 in [8]). In particular, 
there is a critical value s* below which there is only one stationary point, equal to 
the uniform distribution y* = and above which there are other stationary 

points. The critical value is given in [8]) and is equal to 

s* = min mm((I-K)a + KMa)) (26) 

K=1,...,I—1 a>l 

with 4>(x) = f — Wo(— xe~ x ), Wo being the Lambert-W function of index 0. For example 
for 1 = 3, the critical value is s* ~ 2.7456. 

We can apply Corollary [TJ since the state space E = A/ is compact, it follows that 
for s < s*, the stationary distribution given by Eq.( fl4l) . re-scaled by 1/N, converges as 
N — > oo to the uniform distribution. This illustrates the interest of the reversibility 
results in this paper; we do not need to show that all trajectories converge to the 
single stationary point - its uniqueness and the reversibility argument are sufficient. 
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